Golang Job: Data Platform Engineer

The opportunity

As a Data Platform Engineer at ING you will be able to influence the architecture and infrastructure design and operations of a global data analytics platform DAP that enables the organization to use data in an efficient and secure manner. This involves the building and maintaining components that allows the consumers of the platform perform data exploration, experimentation and model deployment.

A day in the life of a Data Platform Engineer

As a Data Platform Engineer you are a very crucial part of the team. Without your work, the platform is stagnant. You build components of the platform that enables the users to build scalable pipelines, deploy models in production, perform data discovery and metadata management. You are the one that is always finding new and innovative ways to improve storage performance, resource management for data processing frameworks. CEO’s rely on you to ensure that the entire organization has a secure and scalable data platform at their disposal. Other products are praising you for all the time you save that would have been required to build and maintain a secure infrastructure to host and process data.

When you arrive at work and finish your morning coffee, you sit with your team to understand what new features make the lives of the platform users easier. You brainstorm ideas on how to create Airflow as a service so several product teams can orchestrate their workflows in a standardized way.

You talk to other business users about what kind of additional features would help them in the secure development environment provided within the platform. After a lunch break with colleagues you think of a new way to improve resource provisioning for the applications running in Kubernetes. As a platform engineer at Analytics Engineering, you will never get bored with many experienced and skilled engineers around you, the freedom to innovate and to not only use open source tooling, but also contribute and teach about them.

The team

ING Analytics Engineering team is a team of 150 people (and growing rapidly) that has the mission to make Banking in ING data driven. We do this by combining Big Data technology with Data Science to deliver high value solutions and products for our organization. We work in a fun and creative environment and we’re dedicated to bringing out the best in both each other and our projects.

Roles and responsibilities

• As part of the platform engineering team, design, implement and improve our big-data infrastructure (on premise and in the cloud, Google Cloud Platform). You’re passionate about balancing scale and performance with a pragmatic approach and devising highly automated solutions to guarantee the encouraged level of reliability, performance and quality in multiple environments
• Building data-related generic tools within the area of data discovery, lineage and governance that will be consumed by platform users
• Closely work with other data engineers to expand existing large-scale data systems or implementing new ones to support evolving data models, analytical products, multi-sourced data analysis, reporting capabilities and production machine learning algorithms along with data scientists
• Deliver high-quality code and solutions, alongside Site Reliability Engineers, to improve data delivery, fault-tolerance and seamlessly integrate with the production platform running our customer-facing products
• End-to-end responsibility on running the data platform, alongside data and SRE teams, monitoring key metrics around availability, performance and data delivery. Drive incidents resolution and root cause investigation within postmortems establishing a culture of collaboration between teams
• Focus on delivering fully automated, tested and self-healing systems. Engage in capacity planning and explore solutions to increase the efficiency of the platform using the opportunities of the open source technologies

Our Stack

Within the Wholesale banking Advanced Analytics team we use (amongst others) the following technologies and frameworks:

Execution:

• Docker / Kubernetes / Openshift

Processing:

• Apache Spark
• Presto/Trino
• Apache Airflow
• Volcano
• Alluxio

Storage:

• S3 (Object storage)
• HDFS
• Apache Kafka
• PostgreSQL, SQL Server and other RDBMS
• Elasticsearch
• Apache Cassandra (and other noSQL DBs)
• Apache Avro, Parquet and other file formats

Programming:

• Python
• JVM based languages (Java/Scala)
• JavaScript
• Go

Data Security and Governance:

• Apache Atlas
• Amundsen (LF)

Data Visualization:

• Apache Superset

How to succeed

We hire smart people like you for your potential. Our biggest expectation is that you’ll stay curious. Keep learning. Take on responsibility. In return, we’ll back you to develop into an even more awesome version of yourself.

• 4+ years of hands-on experience in delivering highly distributed, large scale (data) platforms preferably using open-source technologies
• Programming in scripting languages, e.g. Python
• Programming in a statically typed language, e.g. JVM (Java/Scala/Kotlin), Golang
• Strong experience with SQL/NoSQL datastores (Hive, Cassandra), and other distributed systems (Spark, Kafka, HDFS…)
• Containerization - orchestration/deployment and provisioning automation tools e.g. Docker, Kubernetes, Openshift, CI/CD
• Experience with Implementing continuous delivery and deployment solutions (Gitlab, Azure DevOps)
• Bash scripting

Nice to have the following skills:

• Experience in working on cloud/hybrid solutions
• Security, authentication and authorisation (LDAP / Kerberos / PAM / IAM)
• You have experience with applications configuration management (Helm Charts, Terraform)
• Experience in designing, building, scaling and monitoring data infrastructure
• Maintaining hosted Jupyter/Zeppelin environments
• Experience in working with/contributing to Open-Source projects
• Affinity with Advanced Analytics, Data Science

Rewards & benefits

We want to make sure that it’s possible for you to strike the right balance between your career and your private life. You can find out more about our employment conditions at https://www.ing.jobs/netherlands/Why-ING/benefits.htm

The benefits of working with us at ING include:

• A salary tailored to your qualities and experience
• 24-27 vacation days depending on contract
• Pension scheme
• 13th month salary
• Individual Savings Contribution (BIS), 3.5% of your gross annual salary
• 8% Holiday payment
• Personal growth and challenging work with endless possibilities
• An informal working environment with innovative colleagues
• Work Agile, so new ideas come to life faster

Questions?

Contact Stanley Waccary at [email protected] for more information about this role. Want to apply directly? Please upload your CV and motivation letter by clicking the “Apply” button.

About us

With 57,000 employees and operations in approximately 40 countries, there is no shortage of opportunities for people with initiative who want to help people take a step ahead in life and in business. Do you want to work at the cutting edge of what’s possible and at the same time ensure you work with integrity and hold the customer’s interests at heart? Do you want to be surrounded by progressive, inspiring, diverse and supportive colleagues? Then there is no better place to invest your talents than at ING. Join us. Apply today.